VAC-US-D1.3.7 


DEVELOPMENT OF A NASA INTEGRATED TECHNICAL 
WORKFORCE C AREER DEVELOPMENT MODEL ENTITLED 

REQUISITE OCCUPATION COMPETENCIES AND KNOWLEDGE — THE ROCK 

Robert J. Merit ad 1 

NASA Goddard Space Flight Center, Greenbelt, MD, USA 
Rober t J, M en rad @ nasa. go v 

Dr. Wiley J. Lai son 2 

Stevens Institute of Technology, Hoboken, NJ, USA 
Wi ley Larson @ comcast.net 

ABSTRACT 

I'Jiis paper shares the finding of NASA's Integrated Learning and Development Program (ILDP) jn its effort to reimigorate the HANDS- 
ON practice of space systems engineering and projcct/pragram management through focused couraework, (raining opportunities, on-the- 
job learning and special assignments. Prior lo March 2005. NASA responsibility for technical workforce development (the program/ 
project manager, systems engineering, discipline engineering and aH.-aidaicd communities) was executed by l wo parallel organizations. In 
March 2005 these organi nations merged. The resulting program- ILDP— was chartered lo implement an integrated competency- based 
development model capable of enhancing NASA's technical workforce performance as they face tire complex challenges of Earth science, 
space science, aeronautics and human spaceflight missions. Results developed in collaboration with NASA Field Centers are reported on. 
This work led to definition of the agency's first integrated technical workforce development model known as the Requisite Occupation 
Competencies and Knowledge (the ROCK). Critical processes and products are presented including! "validation" techniques Lo guide 
model development, the Design- A -CtJrriculuM (DAC'UM) prneuss, and creation of the agency’s first systems engineering body-of- 
knowledge. Findings were validated via nine focus groups from industry and government, validated with over 17 space-related 
organizations, at an estimated cost exceeding $300,000 (US). Masters- lev cl programs and (raining progrants have evolved m address the 
needs of these practitioner communities based upon these results, The ROCK reintroduces rigor and depth to the practitioner's development 
in these critical disciplines enabling Cheir ability to take mission concepts from imagination to reality. 
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INTRODUCTION 

The imgic loss of (lie Space Shuttle Columbia on Fehnuiry 1 , 2003 
ushered in an era of significant organizational change and renewed 
workforce commitment at NASA. Guided by die 
recommendations codified in the linal report of the Columbia 
Accident Investigation Board [CAIB, 2003 f 1 agency personnel at 
all levels sought ways in which Lhe in Lent of those 
recommendations could be implemented in a manner that reduced 
tbs in herein, risk associated with human and robuUc sjatcc fligliL Lo 
the lowest level possible. No process, procedure or performing 
group was immune to the scrutiny stemming from the questions of 
"What are we currently doing?”, "What can we Lx: doing belter?’' 
and "I low can we extend improvements across all activities?" 

C farcer development of the technical workforce— (he combined 
communities of project/program managers, systems engineers, 
discipline engineers, :md associated professionals — was one of 
Lite areas reviewed for improve mein during this period- Prior to 
the post-Columbia era, agency- wide development of the 
technical workforce at NASA was managed by two related but 
separate organizations, each with their own tailored programs. 


For prog ram, 1 project management and systems engineering 
these were the Academy for Program/Project leadership 
(A PPL) and NASA Engineering Training (NIT), respectively 
These organizations were located in (he human resources 
organization and their programs were loosely coupled. 

Alter the tragedy, reorganization activities within Lhe agency 
focused upon the Office of the Chief Engineer (OCE). Led by the 
NA.SA Chief Engineer, (he OCE ensures that missions are 
planned and conducted with sound engineering practices, as well 
tts include proper controls and management approaches. The 
' '( T. ; icc i : i : i p i-dics l 'i i I l-’ou^'i "die ami.ti:v of poll; v, 1 1 >. : -.i ■ hi 
and assessment of the engineering and project/program 
management communities. A survey of agency-related activities, 
including development activities, was conducted and resulted in 
the decision that ail technical workforce "development” 
activities shottld be. transferred to the OCE to complement the 
"execution” activities already associated with the office. This 
resulted in the organizational reassignment of APPLand NET to 
die OCE where they joined the NASA Engineering Services 
Center (NE5C) Academy— the training program focusing on the 
development of disci pi inc engineers. 
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It is important to note that while this action resulted in the co- 
location of ihe development programs for p ro g raj li 1 ’ project 
inaongeineiil . vyslcms engineerin': ami discipline o 1 1 |j L n ee l i ll g it 
did it ot consolidate development of the entire technical 
workforce into the OCE. Development of the workforce 
responsible for mission assurance and safely remained within 
the Office of Safety and Mission Assurance. That said the 
integrated workforce development model discussed herein is 
believed Lobe fully extensible to this critical agency community. 

Willi transfer of A PPL and NET to Lite OCE complete, then 
NASA Chief Engineer If. D. G eve den recognized the potential 
duplication of effort and lost potential associated with having 
three parallel development programs to meet the needs of the 
technical workforce at NASA. If NASA was to fully realize its 
goal of meeting the challenges of the post- Columbia era then a 
new approach was required, 'litis resulted in the immediate 
merger of A PPL and NET into one, integrated program in 
March 2005-The new program was tilled the "Integrated 
Learning and Development Program” (ILDP) and 
adrim-isLia lively placed Under the oversight of then OCE 
Division Chief CI'li iso, I Johnson whose leadership faci I i Oiled 
the positive environment within which tins activity was 
conducted. This paper presents the results of 1LDP activities 
front March 2005 to February 2006, 


From its inception, the I LDP team recognized the unique 
opportunity before it. The environment at NASA Headquarters 
and its Field Centers was uncommonly conducive to accepting 
a new integrated approach for the development of the agency's 
technical workforce This opportunity was captured in the l LDP 
Vision Statement: 

"Through a dedicated team, I LDP enables the .VASA 
technical workforce to execute (he agency's space flight 
mission requirements using the proficiency obtained through 
classroom, conference, and targeted work assignments. 1 ’ 

Table I presents the ILDP Need, Goals and Objectives. 

The ILDP team was composed of NASA Civil Servants and 
support contractors, all of whom contributed to its success. The 
following list of personnel represents the key leadership 
assignments; 

- Robert J. Men rad, ILDP Program Manager 

* Anthony J. Mature, ILDP Deputy Program Manager 

* Dr. Wiley J. Larson. ILDP Deputy Program Manager for 
Curriculum Development 

■ Dr. Linda Morris, Area Ixad, Program Project Manager 
Development 


TABLE ]- Integrated Learning and Development Program (ILDP) Need i Goals! / Objectives. The T1 ,DP 
scope reflects the post-Columbia environment intent to optimize career development of the NASA technical 
workforce — the community of program,- project managers, systems engineers and discipline engineers. 


Need: Implement an integrated technical workforce development model to consolidate training where 
possible, provide discipline-specific training tracks when necessary . and pcrmil efficient transfer between 
development tracks to maximize development of the individual arid benefit to the agency. 

t; [.mL 

Objectives 

Provide a unified technical work-force career 

Engage program/projcct, systems engineering and 

development model for tbe agency. 

discipline engineering stakeholders from all 
NASA Centers and Headquarters. 

Model shall address at! proficiency levels. 

The process to create the integrated model will 
employ strategies analogous to those used to 
"validale" readiness for space flight missions foe., 
validate vs. verify). 

Integrate development requirements where 
possible, 

Create or re- validate competency models 

Identify common competencies and common 
proficiency levels for integration. 

Define "cross-over” points between development 
tracks Lo permit participant development in a new 
area w uli minimum redundancy and risk for 
missing content, 

Define curriculum-based course work. 

Offer focused course content in a mariner that is 
immediately applicable to the participant's job. 

Permit tailoring based upon participant post- 

Collect assessment data of job performance from 

course assess menus and performance feedback 
from stakeholders. 

panic ipani and stakeholders. 
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■ Dr. Gary Yale. Area Lead, Systems Engineer Development 

* Dr. Kathleen VanScoyoc, Area Ixad, Practitioner 
Assessment 

The success of this effort was due in very targe part to the 
collaborative environment in which it was executed. The I LDP 
Steering Committee was constituted by merging NET's “NASA 
Engineering Training Working Group” with project.' program 
management development representatives from each NASA 
hi eld Center. Steering committee meetings were conducted 
quarterly, on average, and the level of interdependence and 
cohesion was higher than Lho authors had ever experienced, a 
testimony to the. professional commitment and personal 
dedication each of these steering committee members bi ought 
to the I LDP initiative. Table 2 presents the membership of I he 
[LDP Steeling Committee. 

TABLE 1. The Integrated Learning and Development Program 
Steering Committee. Representatives from all NASA Field Centers 
representing project/program management and systems engineering 
competencies participated in the I LDP Steering Committee. The 
success realised by the I LDP team could noL have been possible 
without their active participation and dedicated efforts. 


Name 

NASA Affiliation 

Vis. Junilla Applin 

Langley Research Center 

Mr Al Bowers 

Diyden Flight Research Center 

Mr David Coote 

$ tennis Space Center 

Mr. Hector IDelgndo 

Kennedy Space Center 

Mr Martin Farktwh 

Glen n Research Center 

Dr. Wiley Larson 

Facilitator 

Ms Sally A rttt Little 

Marshal 1 Space Flight Center 

Mr. Anthony Mature. 

Headquarters 

Mr. Robert Mcnrad 

Headquarters 

Mr. Jerry MuLenburg 

Arties Research Cents r 

Dr, James Ortiz 

Johnson Space Center 

Ms. Barbara Pfarr 

Goddard Space Right Center 

Mr, Jay Schmuecker 

Jet E^opuision Laboratory 

Ms Susan White 

Johnson Space Center 

CREATING T 

HE REQUISITE OCCU P AT 1 0 N AL 

COMPETENCIES AND KNOWI .EDGE (THE ROCK l 


With its vision, need, goals and objectives defined the JLDP 
team turned its attention to the formulation of the integrated 
development model. 

D efin i tion of the Problem 

It is well understood that combined skills and experiences 
necessary within a space mission project team arc both diverse 
and detailed IAPMSS.200S] 4 ; no one competency set has all of 
the ingredients for mission success. Therefore, to be fully 
integrated any technical workforce development model would 
need to represents significant set of skills and experiences. The 


ll-OP team decided to focus initially on program/project 
management and systems engineering. There were several 
reasons for this selection: 

■ Program/project management was identified by the agency 
as a critical competency; 

* NASA was already recognized as a world-class leader in the 
definition of space mission project management competencies 
and this represented a "control" against which results could be 
evaluated against a known and vetted baseline; 

* Systems engineering was identilied by the agency as the 
critical engineering competency; 

* Doth competencies represented senior-level positions that 
heavily influence the overall success of any space mission 
activity; 

* B y selecting only two competencies at the start the [ LDP learn 
was able to simplify the initial data to be ingested by an 
integrated workforce development model thereby mitigating 
complexity risk during the development of the model itself 
and avoid the potential need to descope the effort mid-stream; 

* The ability to efficiently incorporate additional 
competencies in the future would represent validation of a 
key requirement of the model . . .to be a framework permitting 
future growth (scalability) and adaptation (extensibility) 
versus a model representing a point-solution. 

A survey was conducted to identify competency sets already 
developed by each community which could serve as candidates 
for I LDP use. The survey results confirmed that this effort was 
to benefit significantly from the work already completed in 
competency definition for program/project management; a 
major input into the unified model. Both the Project 
Management Institute (PM!) developed Project Management 
Body of Knowledge (PMBOK) [PM I, 2004 1 5 and the NASA 
Academy of Program/Project Leadership (A PPL) competency 
sets represented vetted products appropriate for benchmarking 
by the 1I.DP effort. Since ihc APPL product included the 
additional elements specific to space mission project 
management (e.g. r mission assurance and safety) it was selected 
as the project/ program input into the integration activity, 

A similar survey for systems engineering competency sets was not 
as successful. In fact, the team was unable to identify a complete 
set with sufficient maturity, reviewed by the community or 
applicable to space mission projects in particular. Therefore, the 
IL DP management accepted the derived requirement to ensure 
that the integration mode] definition process captured a vetted 
“systems engineering” competency set as one of its products, 

Formulation— The Paradigm and Preliminary Design 

During formulation of the ROCK the underlying paradigm was 
defined in response to the requirements. The JLDP team 
condensed the conceptual design of any integrated workforce 
development model into three basic categories or phases: 
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■ Global Community Development the set of development 
goals that aim to address the need for universal “awareness". 
Such aspects treat the technical workforce as a single entity 
with a need for common information. 


■ Efficiently capture and integrate this aggregated knowledge 
in a generic and repeatable manner to permit cross- 
referencing across other disciplines with minimized 
opportunities for translation errors. 


* Cross- 1 ’ornmuriiLy I >evelopineni — t he set of development 
goals that two or more communities share in common that 
aim to address the need for "application” knowledge. Such 
aspects treat the involved communities as a single group for 
the purposes of delivering education and training 
opportunities. 

* Hocused Community development — the set of development 
goals ^peeific to one community and not shared b> any other. 
Sneh aspects focus exclusively on that community and tailor 
the education and training cpv ironment lo its specilie needs. 

Figure 1 illustrates the conceptual design of the integrated 
development model or "system" as developed by the I LDP team. 


Regie HI of 
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FIGURE i. The Integrated Learning and Development 
1* ingrain's Integrated Development Model Ciiiio:|itiLLl Design. 

'Die ILDF model Identifies three hasic phases offered ELS essential to 
anj multi -community irttegmion effort: Global Community Training, 
Cross Community I’ mining and Focused Community Training. 


After a survey of available tools one stood out as unique in its 
proven ability to meet these requirements the Design- A - 
CUrriculuM (DACUM) process. During its evaluation, the 
DACUM process was found to have a positive track record both 
inside and outside of NASA making it. Iwth a proven and 
repeatable technique for obtaining the very information from 
experts required by the I LDP team. In preparing for and 
executing the implementation phase of the integrated model's 
development, I LDP was most fortunate in obtaining the active 
participation of Mr. Mark Goldman, NASA Goddard Space 
Flight Center, an ex pen DACUM facilitator whose insight and 
experience proved exceptionally valuable for mitigating the 
inherent risks associated with any process of this kind. 

Details about I he DACUM process arc available from multiple 
sources so will not be repeated here but the following is offered 
as a brief summary. The DACUM process is an occupational 
;i:k.hu.h.h merit tool comprised of ihrcc phases: needs assessment, 
data gathering workshop, and curriculum development. It is a 
controlled, repeatable process. During the facilitator- led 
workshop phase a group of experts in a specific occupation is 
assembled. By the completion of the workshop these experts 
have identified all ihe. areas they arc responsible for in the 
execution of their job fi.e., scope), the tasks they are responsible 
for within those areas (i.e., depth of responsibility), and when in 
the ir career they developed these skills (i.e., development 
timeframe), Thu captured information is expressed in 
behavioral terms for performing the occupation, noi in career 
development terms. This is the key point, the experts always 
focus on the occupation they arc experts in and not in the area 
of workforce development where they may have little or no 
experience. Ihe translation of the Workshop dala into 
turn tar] iim -development terms is completed by a second set of 
experts, As a result, it is the value of this expert-validated date 
efficiently passed between the appropriate expert groups which 
give the DACUM process its power. 


A key component of the formulation process going forward was 
the objective ter employ a tool that supported the principle of 
“mission validation” analogous to that applied by the agency to 
its human and robotic missions. This mettnl ihe tool would be 
required to _ 

■ Elicit l lie accumulated experience of personnel recognized 
as performing at the highest level of proficiency' 

* Prioritize the gross set of responsibility areas and specific 
tasks required of the individual performing at (bis level; 

■ 'lake, into account the minor variances in experience and 
knowledge of any single individual despite the fact they were 
recognized experts in the held: 


The DACUM process is typically used to focus on a single, 
specific occupation. The I LDP team hypothesized that a 
modification to the appliimiiim of the DA( ’l )M too] would make 
it an equally powerful tool in any effort to create an integrated 
development model across multiple occupations. I his minor 
modifieaii on was i inplemented and proved lo be a breakthrough 
for generating the results to follow. The catalyst for identify ing 
this modification came from the conceptual design of the 
integrated workforce appmaeh it sell', 

In defining its ILDP-uniquc execution plan the team identified 
die requirement to execute r/rrec DACUM workshops for ihe 
two occupations focused upon (i.c., project/program 
management and systems engineering). DAT’ I )M workshops I 
and 2 would focus exclusively upon one occupation each. 
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DA Cl M workshop 3, however, would be concluded hvii.li a 
single group of experts spanning both occupations. In this third 
workshop shat the experts woo Id focus on: 

* Identifying the "common" areas of responsibility and 
associated tasks from the data collected in die first two 
DACUMs; 

* Commoiuilitv in each occupation's traditional development 
j wills for achieving the required level of proficiency; 

* Collating each occupation's tasks into the development time- 
bljts of: early- „ mid- and J ate- career timeframes. 

In effect, the I LDP team effectively treated the project program 
and systems engineering domains as a single domain. With the 
occupation-sped fie information defined in dte first two 
workshops, this third workshop defined the common threads for 
hoi 1 1 occupations in terms of what proficiency levels are needed 
and when. The I LDP team recognized the opportunity with this 
third workshop to add new experts from both occupations. In 
this wav "fresh eyes” verted the inputs from tits earlier 
workshops during its execution. The expert panel for the third 
workshop resulted in new observations that improved the 
overall quality of the earlier products but did not result iu die 
identification of any gross errors thereby validating the quality 
of the original inputs. 

] .lafil Integration urn I Validation 

The [LDP Systems Engineering DACUM WHS conducted June 
I -2, 2CH15, in Potomac, Maryland, and was completed on 
schedule wiih no open actions. Table 3 presents the group of 
systems engineering experts who parliei pitted. 

table 3. The Integrated Learning and Development Program 
DACUM Systems engineer Expert Croup. The 1LDP group of 
systems engineering experts was cam posed of individuals from 
inside and omside NASA in order to insure die broadest level of 
eon&cneus- 


System Engineer DACUM 
Representative 

Affiliation 

Mr, David Brawn, Professor. 
Systems Engineering 

Defense Acquisition 
University 

Mr. Hector Delgado, Deputy Dirccity, 
|[kde|iendcnl Technical Authority and 
Systems Management Office 

WAS A - Kennedy Space 
Centex 

Mr. Dave Everett, Senior Systems 
Engineer 

WAS A - Goddard Space 
Flight Center 

Mr. Richard Fullerton, international 
Space Station Systems Engineer 

WAS A - Headquarters, 
Space Operations Mission 
Directorate 

Mr. John Huff. Systems Engineer 

NASA - Headquarters, 
Exploration Systems 
Mission Directorate 

Mr. Ross Jones. Systems and 
Software Assistant Division 
Manager for Systems Engineering 

Jet Propulsion Laboratory 

Mr. Jerry Lake. President 

Systems Management 
International 


I'ABLK 3- The In legrated Leu ruins; and Development Program 
DACUM Systems Engineer Expert Group, (Cunt.} I he If. DP 
group of systems engineering experts was composed of individual* 
from inside and ouisidc NASA in order to Insure the broadest lewd of 
consensus. 


System Engineer DACUM 
Representative 

Afi i hath in 

Mr. Paul LighlSey. Systems Engineer Rati Aerospace 

Mr. Larry Martin, Systems Engineer NASA - Marshall Space 


Flight Centei 

Mr. A1 Motley. Senior Systems 

NASA - Langley Research 

Engineer 

Center 

Mr, Dennis Rohm Senior Systems 

WAS A - Glenn Research 

Engineer 

Center 

Ms. Janette Stecklein, Systems 

NASA - Johnson Space 

Integrator. International Space 
Station 

Center 

Mr. Paul Wextinski, Senior Staff 

WAS A - Ames Research 

Scientist 

Center 

The I LDP Piojecf'Program 

Management DACUM was 

conducted June 23-24, 2005, ir 

i Colorado Springs, Colorado, 

and was completed on schedule 

with no open actions. Table 4 

presents the group of project management expert* who 
juirtieipated. After the Workshop the 1 LI HP learn c posts -check ed 
the results against ihc APPL projcct/program compel cncies. No 
major changes were found: the set was considered re- validated. 
Table 5 presents the projcct/program corn potency set resulting 
from the first 1LDP DACUM workshop. 

TABLE 4- THe Integrated 1, earning and Development Program 
DACUM PrngrUm/Projeet Manager Expert Crimp. The ILDP 
group of |MOject/pnjgrflm experts was COmjxo»ad of individuals from 
inside and outside NASA in order to insure the broadest level of 

consensus, 


Project Manager DACUM 
Representative 

Affiliation 

Mr. Kenneth Alknis, Projcci 
Manger 

Jet Propulsion Laboratory 

Ms, Elizabeth Citrin, Project 

NASA * Goddard Space Right 

Manager 

Center 

Mr. David Collins, Project 
Manager 

WASA - Kennedy Space. Center 

M r. Lee G raham , Project 
Manager 

WAS A - Johnson Space Center 

Mr. Ron Johnson, Project 
M anager 

NASA - Ames Research Center 

Mr. Robert Leroy, Project 
M anager 

] .(jckbeeti Martin 

Mr. John Rogers, Project 
M anager 

SA1C 

Mr, Thomas Sml iff. Project 

NASA - Glenn Research 

Manager 

Center 

Mr. R. Kern Witcher, Project 
Manager 

NASA - Stennis Space Center 


TABLE 5. The Integrated Learning tmd Development Program (JLDT) Froject/Program Manager Competency Set- The H J>P group 
of experts rc-vaUdaled the NASA competency set used to define I he performance requirements for this critical position. 


The Project Management Competencies 

Competency Area 1: Project Conceptualization 

Competency Area 6: NASA Environment 

u 

Project Proposal 

6.1 

Agency Structure and Internal Goals 

1,2 

Requirement Development 1 

6.2 

NASA PM Fyiocedures and Guidelines 

1.3 

Acquisition Management 

6.3 

International Standards and Political Implications' 

1.4 

Project plan n ins;' 



1.3 

Cost- estimating" 



1.6 

Risk Management’ 



Competency Area. 2: Resource Management 

Competency Area 7: Tinman Capital Management 

2.1 

IT and MIS"* 

7J 

Position Management** 

2.2 

Budget and Full Cost Management' 

7.2 

Recruitment. Hiring and Retention 

2.3 

CapiLal Management 

7,3 

Team Dynamics and Management* 

Competency Area 3: Project Implementation 

Coin potency Area 8 : Safely and M issit in Ajv-hu ranee 

3.1 

Syslepps Engineering’ 

s.i 

Env i rO [“invent and Ecology 

3.2 

Design and Development 

8.2 

Workplace Safety 

3.3 

Contract Management* 

8.3 

M i si ion A ssurunce* 



8.4 

Security 

Competency Area 4: Delivery, Operation, and Closeout 

Competency Aren U: Professional and Leadership Development 

4.1 

Logistics Management" 

9.1 

Mentoring and Coaching 

4.2 

Stakeholder Management 

9.2 

C omm unicaLicm'Dec ision M aki n n ' 

4.3 

Technology Transfer and Communication 

9.3 

1 .eadershi p ' 

Pjxi, 

pfjjbed.- lYsmsitum t o Use. Project Delivery and Operations 

9.4 

Ethics" 

Competency Area 5: Program Control and Evaluation 

Competency Area 10: Knowledge Management 

.5.1 

]Vacking/T rending ot Project Performance 

10.1 

Knowledge Capture and Transfer 

5.2 

EVoject Control* 

1 0.2 

Knowledge Sharing 

5.3 

Project Review and Evaluation 




Compiled via aligning APPL/PMDP Competency Model. NASA Center PM Competency Models (MSFC, ARC, JSCJPL), 
NASA NPR7120-5C PM Requirements (Generic Projects), PMI-PMBOK and ILDP DACUM Tasks, 
t High Congruence: similarly represented in at least 6 of 8 sources. 
t High Congruence: lisLcd m 5 sources; added to the list. 

** Ijjw Congruence: I i sled by 2 or fewer resources aruJ/or not detailed. 


J'hc ILDP tcum's survey for systems engineering competency 
.sets for use in this integration process revealed none of suitable 
maturity nor were any vetted by the NAS A systems engineering 
community Using the outputs from the tailored DACUM 
process an activity was conducted to define a competency set 
for this critical position. Table 6 presents the systems 
engineering competency set created by ILDP team members 
Yale, Morris and VanScoyoe. 

With the completion of the first two occupation-specific 
DACUM workshops the ILDP team had the inpui-s necessary to 
conduct the third integrating workshop. . .the first critical step 
towards realizing a truly integrated development model. To 
execute this workshop ILDP brought together a composite 
group of e.X|>erts who spanned both project, 'program and 
systems engineering. While experts were drawn upon from both 
inside and outside NASA, a bias towards internal agency 
representation was employed in order to insure that the resulting 
products would have suitable buy-in later during the execution 
phase Table 7 presents the group of project, ''program and 
systems engineering experts who participated 


ILDP conducted its combined PM/SE workshop in July 2005, 
in Boulder, Colorado, and was completed on schedule with no 
open actions. As designed, the workshop focused the gathered 
experts in a process to identify ’'common project pro gram and 
systems engineering areas of responsibility, as well as common 
tasks, By definition those responsibilities and tasks that 
remained unpaired were not suitable candidates for integration 
and were categorized as Hocused Training opportunities per the 
definition above. 

The workshop then focused on the categorization of when in the 
career track of a systems engineer or project,' program manager 
the individual needs to be able to execute the requisite tasks 
This categorization was completed using tlirec career 
timeframes: early-, mid- and laie-eureer. The group of exports 
completed the workshop without the need to revisit the original 
process designed by the ILDP team affirming the validity of 
both the tailored DACUM tool and the integrated model 
conceptual design. 
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TABLE 6, The Integrated Learning and Development Program (I LDP) DACUM Systems Engineer Competency Set, The ILDP group of 
experts provided input into definition of this competency set— the hist known product of its kind for the systems engineering occupation. 


Systems Engineering Competencies' 

Competency Area: 1,0 Concepts and Architecture 

Competency Area: 6,0 NASA Internal and External Environment 

l.l 

Mission Needs Statement 

6. 1 Agency Structure, Mission, and internal Coals 

3.2 

System Environments 

6.2 NASA PMfSE Procedures and Guidelines 

13 

Trade Studies 

6.3 External Relationships 

E .4 

System Architecture 


Competency Area: 2,0 System Design 

Competency Areat 7.0 Human Capital Management 

2.1 

Stakeholder Expectation Definition and Management 

7. 1 Technical Staffing and Performance 

2 2 

Technical Requirements Definition 

7.2 Team Dynamics and Management 

2.3 

Logical Decomposition 


2.4 

Design Solution Definition 


Competency Area: 3,0 Production, Product Transition, 
Operations 

Competency Area: 8.0 Security, Safety and Mission Assurance 

3 , 1 

Prod uct Implementation 

8.1 Security 

3.2 

3.3 

Product Integration 
Product Verification 

8-2 Safety and M is si on As.su ranee 

3.4 

Product Validation 


3,5 

Product Transi tion 


3.6 

OpGiati On. 4 ; 


Competency Area: 4,0 Technical Management 

Competency Area !>: Professional and Leadership Development 

4,1 

Technical Planning 

9. 1 Mcrnori ng and Coachi ng 

4.2 

Requirements Management 

9.;2 Communication 

4.3 

interface Maruigcmcn t 

9.3 Leadership 

4.4 

Technical Risk Management 


4.5 

Conti gurati on Management 


4.6 

Technical Data Management 


4.7 

Fee hair ai Assessment 


4.8 

Techni Ceil Deci s ion A nal yai^ 


Competency Area; 5.(1 Project .Management and Control 

Competency Area: 10,(1 Knowledge Management 

5.1 

5.2 

5.3 

Acquisition Strategies and Procurement 
Resource Management 
Contract Management 

[0. i Knowledge Capture and Transfer 

5 4 

Systems Engineering Management 



* Compiled via aligning of SE CompetcncicFJPmccssc^EunctioitfTask from Centers (MS FC, GSFC, JSC, K SC, ARC), NASA (ILDP SE 
DACUM; OCE Systems Engineering NPR (Draft)- APPL PMDP Com|jelencv Model; NESC), and other external SE Sources (DOD. 

I NCOS F.). 
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lABI.t. T. 3 he Integrated l earning unci Development Program 
DACUM Combined PM/SE Expert Group. The I LDP group of 
project: program and systems engineering experts, was composed of 
individuals primarily from inside NASA in Order to insure consensus 
on this innovative way for technical workforce career development. 


Combined PM-KE 

DACUM 

Representatives 

Affiliation 

Mr. Moses Adofco 

Consultant 

Mr Stephen Kapurch 

NASA • Headquarters. Office of ibe 
Chief Engineer 

Dr. Wiley Larson 

Consultant 

Mr. Ken Ledbetter 

NASA - Headquarters, Space Science 
Mission Directorate 

Mr Anthony Mature 

NASA - Headquarters, II, DP 

Mr. Robert Me mad 

NASA Headquarters. ILDP 

Dr Linda Morris 

Consultant 

Dr. James Ortiz 

NASA * Johnson Space Center 

Ms. .lonettc Sleek luin 

NASA - Johnson Space Center 

Dr. KaLhleen VanSeuyoc 

Consultant 

Mr, Carl Wales 

N A S A - Goddard S pace FI ighl Center 

Mr, R. Kem Witcher 

NASA - Sierras Space Center 


The categorization results for each occupation repealed that the 
distribution of tasks to he mastered as a function of time couid 
lie described as Gaussian in nature. Interestingly, the median tor 
a systems engineer's development was observed to be in the 
mid-career period By contrast the median for a projecti'prognun 
managers development was observed to be in the I ale-career 
period. The I LDP team discussed these observations with the 
assembled experts to determine its importance and realized that 
the data had revealed the typical career path for individuals at 
NASA: generally speaking, individuals begin their career as 
discipline engineers then transition to systems engineers in the 
middle and then move onto pcoj&cl/pnPgram management 
positions later. Further data analysis determined that, this 
workshop had also provided the needed information on how to 
approach the reality of individuals changing jobs as they spend 
time in the organization. This proved very useful m approaching 
the goal of identifying places in tile II. DP development model 
analogous to a university student changing undergraduate 
majors; some courses will count inwards the requirements for 
the new major but additional mid-level (i.c., level 200} courses 
may be required as a consequence of the change By ihe end of 
tire workshop the participants recognized the unique results that 
the team had generated. Table 8 presents the findings of this 
third DACUM. fills data documents the significant overlap of 
the capabilities needed by project managers and systems 
engineers as documented in the center column, 


TABLES. The Integrated Learning and Devs/tnpim-m Program 1 1 3, HP) DACUM— Integrating Systems Engineering and Project 
Management Dtwclnpmenl — Workshop Results. The focus here is Lhe set of shared capabilities beLween svslcms engineers atld pfO|CCt 
managers. Nole that n signtficanl number of capabilities are common. The systems engineers lend to focus on technical integrity . while the project 
manger must focus on Lbe project's overall technical performance, cost, schedule and risk. 


Primary Systems Engineering Typically Shared I'M and SE 


Primary Project Management 


Cnnccpts and Architecture 

Form Mission Need Statement 
Describe System Environments 
E\;rforin Trade Studies 
Create System Arc hi Lectures, 

System Des ign 

Qeb ne/Manage Technical 
Siakehcdder Exjxciations 
Define Technical Requirements 
Logically Decompose System 
Design Systems 
Technical Management 
Plan Technical Effort 
Manage Requirements 
Manage Interface* 

Manage Technical Risk 
Manage Configuration 
Manage Technical Data 
Manage Decision Process 
Production, Trar sitiotn Operations 
Implemcnl the Pioduci 
Integrate Systems 
Verify the System 

Validate the System 
Transition the System 
Conduct Opcralioris 


I eader s hip DciQnpnu nt 

Coach and Mentor Ptotegps 
Communicate Effectively 
J .cad Teams and Organizations 

Proj ect Man a gfmcnlj't Vuit inl 

Oversee Technical Acquisition 
Manage Resources 

1 Illiii an Capita] 

Staff Technical Organization 

Oversee/Manage Performance 

Manage Team Dynamics 

O run niz uLi a nal Ivn vi n inmenl 

Understand Organizational Goals 

Apply PM/SE Guidelines 

Manage External Relationships 

Security. Saf ety and Mission Assurance 

Organize Security. Safety and Mission Assurance 

Kmmlcdge Vlamiacn u nt 

Capture Knowledge 

Organ i f.c: Diitn bule- Knutv ledge 


P roject Conc eptualization 
Prepare Project Proposal 
Develop project Requirements 
Manage Acq ui si Lion 
Plan the Project 
Estimate Lifecycle Cost 
Manage Project Risk 


Resource Mnnuecmcot 
Establish and Manage IT and IS 
Manage Budget and Full-Cost Account 
Manage Capital Assets and Investment 
Project 1 m plcnientation 
Manage System* Engineering Effort 
Oversee Design and Development Activity 
Manage Contracts 


Manage Stakeholders 
Manage Logistics 

Manage Tec hnology Transfer and Comm 
Manage Transition. Delivery and Qps 


Project Control and .Kyultralicin 

Track and Trend Project Performance 
Pcrfoim Project Control 
Organize Manage Project Reviews and 
Evaluation 
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With die DACUM workshops completed the [LDP team 
l I l m r ] jj ad its focus from formulation to implementation; the 
creation of the integrated workforce development model (i.e. T 
the R(K'K) using the data collected. A 4-lier development 
framework for I he model was chosen, This decision was made 
for several reasons: 

* The A PPL projcct/program competency set inherited by 

I LDP was a 4-tier framework and no requirements to modify 
it was identified during the workshops; 

* 4-i ier frameworks have been employed by other groups 
outside NASA thereby y ielding an opportunity for 
consistency 1 to be maintained; 

* A framework with Lite minimum number of required tiers 
was considered optimized since tier-specific overhead 
activities would be reduced to the lowest level practical; 

■ Discussions wjih the stakeholders confirmed that the tiers 
categorized the spectrum of performance desired from the 
workforce in fewest terms. 

The 4-tierS, referred to ns performance levels, selected for the 
ROCK are; 

* K nOw — the- devc I opancnl object] V e for tier M 1 is 1 'awa re ness” 
and. is calibrated to the “team member". To successfully 
complete the criteria an individual must he able to 
demonstrate an understanding of I he organization at the 
agency, center, directorate and division levels 

* Apply the development objective for tier it 2 h "hands on” 
and is calibrated to tin? 'leader of a small activity”. To 


successfully complete the criteria an individual must be able 
to demonstrate their ability to develop small space mission 
systems or projects. Significant personal involvement with 
l he work being done is assumed. 

- Manage— the development objective for I ier #3 is “to get 
work done through people” and is calibrated to the "lender of 
a large activity". To successfully complete the criteria an 
individual must be able to demonstrate their ability to lead a 
large-scale engineering team or project. At (his point 
personal involvement with the work being done is minimal 
and the individual is managing the teams that are completing 
the work. 

■ Guide — the development objective for tier 04 is “to get work 
done through projects” and is calibrated to the "leader of a 
system of systems”. To successfully complete the criteria an 
Individual must be able to demonstrate their ability to lead a 
group of systems engineering teams or discrete projceSs. 
Here the indiv idunl is defining strategic vision for the 
organization and involved m knowledge sharing activities 
(c.g., Masters Horums). 

The ILDP team translated performance goals into an overall 
curriculum for proj c c i/p ro g ram managers and systems engineer 
dev elopment using the 4-Lier framework described above and the 
products from ibe DACUM workshops. The curriculum was 
pursed into discrete courses with the target audience defined for 
each. J 'he completion of this activity represents i he agency - first 
integrated technical workforce development model addressing 
the career development requirements of project.' program 
managers and systems engineers. Hgurc 2 presents tbe Requisite 
Occupation Competencies and Knowledge created by ILDP 


f 





'//////, 


hiteeraled [jcnnung and Development 


I riU-rnK'di-cUe Prnjal Management 


Intermediate S;. Mem \ nginceriiq 


e -T : ^^^4* ja g j jfeiffi ; 

— . - 


•w CVinununicaiions A Leadership II Risk Management II 

vXttyyffi/y//y//yyyyyyyyA fyyyy/yyy^/AyyYyyyyyyyyyyyyyyyyy^ 

biLrLh.Euctii.nl lu N \SA — \ isiort. Missions and Si sterns 

K boundlilU<us in E .ugtueering. SjMciii I agirievring & Project Management 

*/}■// , ■■■ ■.. • /, '//////// s/s/s //s/- > 
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Level -4 
( 'miiplcu.' 


Ia.il- I t 
(Limplctv 


ljjvei-2 

CunijiRu? 


E jcvcI- i 
Complete 


f ore f "in i im,-* 


. _ „■ Candidate cmniTumilies 
for fut lire integral ion 


t Shaded) Disciplinc 


niJlItf! 2. The Integrated I earning anil Development. lYiigrain I'll. DP) Krijuiiril Occupation CriinpetcncicSi and kilim- ledge Program 
(tht ROCK). This version of the ROCK integrates the career development requirements of NASA p njj ee t/prog ram managers and systems 
-engineers, as well ns establishes the methodology for adding new communities such as discipline engineering or safety and mission assurance at a 
Later dale- Note: “Cmss-Community Development'' as defined herein only occurs when three or more communities arc integrated into the model. 
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In detailed form, the ROCK identifies actual course modufes for 
each performance level, sorted by targeted community, in order 
Lo permit detailed planning of the pari id pant's development 
program. 

EXTERNAL VALIDATION ACTIVITIES 

External organizations have been involved in the validation of 
the systems engineer capabilities since the I LDP SE 
competency set was developed and validated within the NASA 
community- To date. 1 1. organisations have participated in a 
validation activity that foctises on performance level 2 
application. In this validation activity, r e p resentatives From 
participating organizations have been presented wilh the ILDP 
SE capabilities and asked to categorize each capability as eilher 
CRITICAI to the development of their systems engineers, 
NECESSARY but not critical, or OPTIONAL. Organizations 
engaged iri the validation activity include: NASA (Johnson 
Space Center, Amen Research fleeter, Glenn Research Center), 
US Air Force Space and Missile Center, Lockheed Martin 
(Denver), 1 he Boeing Company (Space and Intelligence), Ball 
Aerospace, Deutsches Zentrum fur Left- und Raumfahrt 
(DLR), European Space Agency (ESA) European Space and 
Technology Center (ESTEO, Centre National d'Etudcs 
Spadales (CNES), and Agenda Spa/ink Italians (AS1). The 
[nicrnational Academy of Astronautics study team has vetted 
die capabilities with the SH community in the 1 : S and Humpe. 


To dale, several organizations— within the United Stare.', and 
internationally— have elected to use the ROCK as the 
fou ndatiou to bui Id the i r programs upon . 

* The NASA - Headquarters “Academy of Program 'Project & 
Engineering Leadership" (APPEL) is using the subject 
workforce development model to guide the development of 
its systems engineer leadership development program 

1 The Stevens Institute of Technology has Created st Masters 
level and Graduate Certificate Program in Space Systems 
Engineering based on the capabilities defined in this 
workforce development program. 

* Moth NASA - Ames Research Center and NASA - Johnson 
Space Center Itavc adopted the ROCK as the educational 
component of their system engineering leadership 
development programs, 

* The Technical University of Delft's SpaceTech Masters 
Program in Space Systems Engineering adjusted iLs 
educational components to be in-line with the ROCK. 

Table 9 presents the composite results of this validation activity 
as of August 1,200$. 


TABLE 9. Systems Engineer Capabilities, The left column lists systems engineer capabilities as documented by 
NASA/LLDP in the Requisite Occupation Competencies and Knowledge (the ROCK). The superscript “p" beside 
identifies those capabilities assoc iaied wiLh NASA's procedural requirements in NPR 7123, ) a "NASA Systems 
Engineering Processes and Requirements' 1 . Performance level-2 capabilities were assessed by 17 organ i zaiionss as 
either Critical in SE development. Necessary (but not critical) or Optional. Only the results for systems engineer 
performance level -2 are show n here . 

System Engineer Capabilities] Performance Levels I - IV 


1.0 Concepts and Architecture 
( . I M ission Needs Statement 
1.2 System Environments 
i .3 Trade S Ludies 
l .4 S ysiem Architecture 


i ii m iv 

Aware Apply Manage Strategy 

C-N 
C 
C 
C 


2.0 System Design 

2. 1 Stakeholder Expectation Definition & Management P C 

2 - 2 Technical KequLreinents Dclinilion ^ C 

2.3 Ijogical Decomposition p C 

2.4 Design Solution Definition p C 

3- 0 Product inn, Product Transition and Ops 

3-1 Product I mple mentation p O 

3.2 Product Integration p C 

3-3 PnoducL Verification p C 

3 .4 Prod uct Validation p N 

3-5 Product Tianiii Lion p O 

3.6 Operations N 
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lARf.r'J, Systems l\ [igniter CainlErililiea, (CflULL The left column UhLh systems engineer capabilities as 


documented by NASA/I LDP in the Req uisite Occupation Competencies and Knowledge (the ROCK). The superscript 
L, p" beside identities (hose capabilities associated with NASA's procedural requirements in NPR 7123.1a “NASA 
Systems Engineering Processes and Requirements", performance level-2 capabilities were assessed by 17 
organizations as cither Critical m ST development. Necessary (but not critical) or Optional, Only Ihc results for 
systems engineer perf Druumce 1 evel-2 ure show n here. 

System Engineer Capabilities] 

Performance Levels [ - IV 

I 

J[ III IV 

4.(1 Technical Managr-munl 
4.1 Technical Planning p 

C 

4,2 Requirements Management p 

C 

4.3 Interfiice Management |J 

C 

4A Technical Risk Management 11 

c 

4,5 Cofifiguraliun Management p 

N 

4.6 Technical Data Management p 

O 

4.7 technical Assessment p 

c 

4.H Technical Decision Analysis p 

c 

,\U Project Mini ugenicn 1 and Control 

5.1 Acquisition Strategies and Procurement 

0 

5 ,2 Resource M aimgement p 

0 

5.3 Contract Management 

0 

5,4 Systems Engineering Management f 

N 

6.6 Internal mid External Environments 

6,1 Organization Struct ure, Mission, Internal Goals. 

O 

6.2 FM/SE Procedures and Guidelines 

0 

6,3 External Relationships 

O 

7.0 Human Capitol ManageiiKiH 

7.1 Technical Stalling and Peri'onnnnce 

O 

7.2 learn Dynamics and Management 

c 

ft.OSLeui'ity, Safely and Mission Assurance 
&.\ Security 

o 

82 Safety and Mission Assurance 

N 

9.0 Professional and Leadership Development 

9.1 Mentoring and Coaching 

O 

9.1 Communication 

C 

9.3 Leaderahip 

c 

ll).(l Kmm ledge Management 

10. 1 Knowledge Captu re and Management 

o 


SUMMARY 

Jl is the human condition to respond to tragedy with a renewed 
commitment to excellence. It was in this spirit that II. I3P was 
chartered by ihc NASA Office of Hie Chief Engineer. Here we 
have presented the 1 LDP activities that resulted in; 


* Re -validated competency set for projccl/program 

management and the introduction of a validated competency 
set for system^ engineering- 

■ An integrated competency set for projecD'program managers 
and systems engineers; 


A conceptual model for integrated workforce development 
spanning three categories of development; 


El 


* A 4- tier integrated development model entitled ihe ROCK, 

Since tlie completion of tins work resources continue to be 
e.\ pended on addressing ihe development needs of the systems 


engineering community based upon the ROCK, k is a critical 
position on any project lean and the efficient development til' 
[hi* community represents a wide -spread need both inside and 
outside the aerospace sector (Larson and Hill. 2008 ] 6 . "I lie 
RCX'K is tin internationally recognized technical workforce 
development model. Several organizations are adopting the 
ROCK as [he foundation for their systems engineering 
development program. At least one Masters level certificate 
program in Systems Engineering, based upon the ROCK, is now 
«v nibble in the United Slates of America. 

I he ROCK offers organizations considerable flexibility when 
addressing their technical workforce career development 
requirements. In instances where a single occupation will be 
focused upon {e.g„, systems engineering), the ROCK provides a 
consistent and unified 4-tier development framework to guide a 
participant's development, When organizational need* require a 
framework for multiple technical occupations the ROCK 
continues to offer & suible framework to meet those needs. 
Hence, we believe the ROCK offers any organization u unique 
platform of stability for addressing technical workforce 
development requirements in a dynamic : environment. 


This paper would not be complete without calling attention to 
the contribution of ihe professionals who made up the ILDP 
Team and its support contractors. It was through their 
professional commitment and personal dedication that E hi 5 > 
work was accomplished. Special mention is made for Dr. Linda 
Morris. Dr. Kathleen VanScoyoc, and Dr. Gary Yale whose 
efforts resulted in the development of the Systems Engineering 
competency set presented herein. 


1 Mr, M pii rad served as J LDP Program Manager during a one -year cfctai I to 
NASA Headquarter. Washitijtion. D.C- 

O Lantan served as [LDP Deputy Program Mais*ger - Curriculum white 
under contract So NASA Headquarters. Washington. D.C. 
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